home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
Tech Arsenal 1
/
Tech Arsenal (Arsenal Computer).ISO
/
tek-13
/
amac44b.zip
/
FIL002.QM
< prev
next >
Wrap
Text File
|
1992-05-27
|
23KB
|
543 lines
* fil002.qm
* Macros To Mark or Remove Duplicate Lines In Files
* Written By Tom Hogshead
* [ See FILExx.QM For Use ]
* 5/30/92
* Key Description Bytes Secs Speed
* ───── ─────────────────────────────────────── ───── ──────────
* @5 - Remove Duplicate Lines, ( ) 145 5.6 a
* Duplicated Lines Retained In Nul File
*
* @7 - Retain Only Non-Duplicated Lines and 90 7.7 b
* Remove All Duplicated Lines
*
* @8 - Retain Only Non-Duplicated Lines and 162 8.3 b
* Remove All Duplicated Lines,
* Duplicated Lines Retained In Nul File
*
* @f8 - Retain Only Non-Duplicated Lines and 141 9.0 b
* Remove All Duplicated Lines,
* Duplicated Lines Retained In Nul File,
* Modification of @8 Using Scratch Buffers
*
* @9 - Mark Duplicated Lines 104 14.2 c
*
* @0 - Mark Non-Duplicated Lines 141 15.9 d
*
*------
* In a 200 hundred line file on a 12 mhz AT, times in seconds to:
* a - Remove 90 duplicate lines
* b - Remove 180 duplicated lines
* c - Mark 180 duplicated lines
* d - Mark 20 non-duplicated lines
*
* {e:\up\FILE*} Return To FILExx.QM
*
*--eoi
* DESCRIPTION
* ------------
* The following set of macros remove and/or mark non-empty duplicate
* and duplicated lines in the current file. These macros are in
* Amac43.zip fil002.qm. Following describes the macros' functions.
* The following four macros remove duplicate or duplicated lines from
* the current file:
* @5 - Removes all non-empty duplicate lines from the current
* file being edited, leaving only the first line of any
* duplicate lines. All removed duplicate lines are shown
* as a block in the Nul file.
* @7 - Retains only all non-empty non-duplicated lines in the
* current file and remove all lines that are duplicated.
* Duplicated lines are Not retained in the Nul file for
* viewing.
* @8 - Retains only all non-empty non-duplicated lines in the
* current file and remove all lines that are duplicated.
* Duplicated lines are retained in the Nul file for
* viewing. @8 is limited to the size of your UnKill
* Buffer, or a maximium of 300 duplicated lines.
* @f8 - Retains only all non-empty non-duplicated lines in the
* current file and removes all lines that are duplicated.
* Duplicated lines are retained in the Nul file for
* viewing. The scratch buffer technique is used here, so
* the number of duplicated lines that can be removed and
* viewed in the Nul file is practically unlimited.
* The following two macros leave all lines in the file but mark all
* duplicated and non-duplicated lines:
* @9 - Marks all non-empty duplicated lines in the current file
* with '*▐' in column 77 and leaves lines that are not
* duplicated in the file unmarked.
* @0 - Marks all non-empty non-duplicated lines in the current
* file with '*▐' in column 77 and leaves lines that are
* duplicated in the file unmarked.
* A FEW NOTES
* ------------
* - QEdit v2.15 is required.
* - Qconfig unkill buffer should be set to 300.
* - Ascii #173 cannot exist in file macro is invoked in.
* - Macros are irrespective of all toggles/Qconfig settings.
* - Macros @5 and @8 are limited to 300 duplicate(d) lines.
* - Macros @9 and @0 may overwrite text in columns 77-78.
* M A C R O S
* ----------------------------------------------------------------------
* @(5) Remove All Duplicate Lines From File,
* Uses UnKill, Limmited to 300 Duplicated Lines
* ----------------------------------------------------------------------
* This macro removes all non-empty duplicate lines from the current
* file being edited, leaving only the first line of any duplicate
* lines. All removed duplicate lines are shown as a block in the Nul
* file. Press <enter> to return to file with duplicate lines removed.
* The Nul file containing the removed duplicate lines remains loaded
* for viewing.
* NOTE: The number of duplicate lines that can be removed with this
* macro is unlimited. However the number of deleted lines that can be
* viewed in the Nul file cannot exceed 300 because we are limited to
* the size of the unkill buffer set with Qconfig. If the number of
* deleted lines exceeds the size of the Qconfig unkill buffer setting,
* only the last deleted lines equal to the size of the unkill buffer
* will be shown in the Nul file.
@5 macrobegin
begfile
setscreenoff setsoundoff
setwordwrapmode togglewordwrap
* ----------- Clear Unkill Buffer, Insure Nul Not In Ring -----------*
markcolumn editfile "nul" return
CLEAR:
unkill jtrue CLEAR
quit gotoblockbeg
* ---------------------- Bypass All Empty Lines ----------------------*
LOOP:
endline begline jtrue NOT_EMPTY
cursordown jtrue LOOP
NOT_EMPTY:
* ------- Copy Line To Scrap For Find, Insert Position Marker -------*
unmarkblock makectrofscreen
begline markcolumn endline
cursorleft markcolumn
copy
cursorright #173
* -------------------- Find/Delete Duplicate Lines -------------------*
FIND_DUPS:
find
paste return delline return
jfalse NO_DUPS * Del markers when no more dup's
isbegline jfalse FIND_DUPS * Bypass multiple dup's in same line
pasteover gotoblockend * Paste block over line with 'find'
endline jtrue FIND_DUPS * Test if dup line
delline * If false, it's a dup, delete it
cursorup * In case next line is dup also
jump FIND_DUPS * Continue 'til no more dup's
* --------------------- Delete Position Marker ---------------------*
NO_DUPS:
endline * Pos for find *|
find #173 return "b" return
jfalse REMOVED
delch * Remove pos marker
jump NO_DUPS
REMOVED:
cursordown jtrue LOOP * MOVE down to next line and loop
* -------------- Add Line At EOF If Last Line Not Empty --------------*
endline begline jfalse EMPTY
addline
EMPTY:
* -------------- Cut Duplicate Lines In Block From EOF --------------*
unmarkblock
unkill jtrue HASDUPS
"No Dup's"
HASDUPS:
markline
UNDELETE:
unkill jtrue UNDELETE
endfile cut begfile
* ------------- Paste Duplicate Lines To Nul For Viewing -------------*
editfile return
paste
* ----------------- Remove All Empty Lines From Nul -----------------*
CHK4BLANK: endline begline Jtrue NOTBLANK delline Jtrue CHK4BLANK
NOTBLANK: endpara cursordown Jtrue CHK4BLANK
* ------------- Position To View Duplicate Lines Deleted -------------*
makebotofscreen cursorup
setscreenon
pause * Press <enter> to return to file w/o dup's
prevfile unmarkblock
togglewordwrap
*
* 144 bytes Wed 05-27-1992 11:35:18 (TH @5, to ALL #3/854-56)
* 145 bytes Wed 05-27-1992 19:15:35 (TH @5, added endline *|)
*
* ----------------------------------------------------------------------
* @(7) Retain Only All Non-Duplicated Lines In File
* And Remove All Duplicated Lines
* ----------------------------------------------------------------------
* Retains only all non-empty non-duplicated lines in the current file
* and remove all lines that are duplicated. Duplicated lines are NOT
* retained in the Nul file for viewing.
@7 macrobegin
begfile
setscreenoff setsoundoff
setwordwrapmode togglewordwrap
* ---------------------- Bypass All Empty Lines ----------------------*
LOOP:
endline begline jtrue NOT_EMPTY
cursordown jtrue LOOP
NOT_EMPTY:
* ------- Copy Line To Scrap For Find, Insert Position Marker -------*
unmarkblock makectrofscreen
begline markcolumn endline
cursorleft markcolumn
copy
cursorright #173
* -------------------- Find/Delete Duplicate Lines -------------------*
FIND_DUPS:
find paste return * Search for first duplicate line
delline return
jfalse NO_DUPS * Del markers when no more dup's
isbegline jfalse FIND_DUPS * Bypass multiple dup's in same line
pasteover gotoblockend * Paste block over line with 'find'
endline jtrue FIND_DUPS * Test true if no dup line
delline * If false, it's a dup, delete it
cursorup * In case next line is dup also
HAS_DUPS:
endline * Pos for find *|
find #173 return "b" return
jfalse REMOVE_DUP
delch * Remove pos marker
jump HAS_DUPS
REMOVE_DUP:
delline * Line had at least 1 dup, del it
cursorup * In case next line is dup also
jump LOOP
NO_DUPS:
endline * Pos for find *|
find #173 return "b" return
jfalse REMOVED
delch * Remove pos marker
jump NO_DUPS
REMOVED:
cursordown jtrue LOOP
begfile togglewordwrap
*
* 88 bytes Wed 05-27-1992 11:35:26 (TH @7, to ALL #3/854-56)
* 90 bytes Wed 05-27-1992 19:15:44 (TH @7, added endline *|)
*
* ----------------------------------------------------------------------
* @(8) Retain Only All Non-Duplicated Lines In File
* And Remove All Duplicated Lines,
* Duplicated Lines Retained In Nul File,
* Uses UnKill, Limmited to 300 Duplicated Lines
* ----------------------------------------------------------------------
* This macro retains only all non-empty non-duplicated lines in the
* current file and removes all lines that are duplicated. Duplicated
* lines are retained in the Nul file for viewing. @8 uses the UnKill
* buffer. Refer to @5 NOTE for 300 line limitation.
@8 macrobegin
begfile
setscreenoff setsoundoff
setwordwrapmode togglewordwrap
* ----------- Clear Unkill Buffer, Insure Nul Not In Ring -----------*
markcolumn editfile "nul" return
CLEAR:
unkill jtrue CLEAR
quit gotoblockbeg
* ---------------------- Bypass All Empty Lines ----------------------*
LOOP:
endline begline jtrue NOT_EMPTY
cursordown jtrue LOOP
NOT_EMPTY:
* ------- Copy Line To Scrap For Find, Insert Position Marker -------*
unmarkblock makectrofscreen
begline markcolumn endline
cursorleft markcolumn
copy
cursorright #173
* -------------------- Find/Delete Duplicate Lines -------------------*
FIND_DUPS:
find paste return * Search for first duplicate line
delline return
jfalse NO_DUPS * Del markers when no more dup's
isbegline jfalse FIND_DUPS * Bypass multiple dup's in same line
pasteover gotoblockend * Paste block over line with 'find'
endline jtrue FIND_DUPS * Test true if no dup line
delline * If false, it's a dup, delete it
cursorup * In case next line is dup also
HAS_DUPS:
endline * Pos for find *|
find #173 return "b" return
jfalse REMOVE_DUP
delch * Remove pos marker
jump HAS_DUPS
REMOVE_DUP:
delline * Line had at least 1 dup, del it
cursorup * In case next line is dup also
jump LOOP
NO_DUPS:
endline * Pos for find *|
find #173 return "b" return
jfalse REMOVED
delch * Remove pos marker
jump NO_DUPS
REMOVED:
cursordown jtrue LOOP * MOVE down to next line and loop
* -------------- Add Line At EOF If Last Line Not Empty --------------*
endline begline jfalse EMPTY
addline
EMPTY:
* -------------- Cut Duplicate Lines In Block From EOF --------------*
unmarkblock
unkill jtrue HASDUPS
"No Dup's"
HASDUPS:
markline
UNDELETE:
unkill jtrue UNDELETE
endfile cut begfile
* ------------- Paste Duplicate Lines To Nul For Viewing -------------*
editfile return
paste
* ----------------- Remove All Empty Lines From Nul -----------------*
CHK4BLANK: endline begline Jtrue NOTBLANK delline Jtrue CHK4BLANK
NOTBLANK: endpara cursordown Jtrue CHK4BLANK
* ------------- Position To View Duplicate Lines Deleted -------------*
makebotofscreen cursorup
setscreenon
pause * Press <enter> to return to file w/non-dup's
prevfile unmarkblock
togglewordwrap
*
* 160 bytes Wed 05-27-1992 14:52:07 (TH @8, to ALL #3/854-56)
* 162 bytes Wed 05-27-1992 19:15:48 (TH @8, added endline *|)
*
* ----------------------------------------------------------------------
* @(9) Mark Duplicated Lines In File
* ----------------------------------------------------------------------
* This macro marks all non-empty duplicated lines in the current file
* with '*▐' in column 77 and leaves lines that are not duplicated in the
* file unmarked. Since the macro sets Insert Off, any duplicated
* lines having text in columns 77 and 78 will be overwritten. This
* macro operates in reverse of @0.
@9 macrobegin
begfile
setscreenoff setsoundoff
setwordwrapmode togglewordwrap
setinsmode toggleinsert
* ---------------------- Bypass All Empty Lines ----------------------*
LOOP:
endline begline jtrue NOT_EMPTY
cursordown jtrue LOOP
NOT_EMPTY:
* ------- Copy Line To Scrap For Find, Insert Position Marker -------*
unmarkblock makectrofscreen
begline markcolumn endline
cursorleft markcolumn
copy
cursorright #173 * Insert starting pos marker
* --------------------- Find/Mark Duplicate Lines --------------------*
FIND_DUPS:
find paste return * Search for next duplicate line
delline return
jfalse NO_DUPS * Del markers when no more dup's
isbegline jfalse FIND_DUPS * Bypass multiple dup's in same line
pasteover gotoblockend * Paste block over line with 'find'
endline jtrue FIND_DUPS * Test true if no dup line
gotocolumn "77" return "*▐"
HAS_DUPS:
endline * Pos for find *|
find #173 return "b" return
jfalse MARK_DUP
delch * Remove pos marker
jump HAS_DUPS
MARK_DUP:
gotocolumn return "*▐"
jump REMOVED * MOVE down to next line and loop
NO_DUPS:
find #173 return "b" return
jfalse REMOVED
delch * Remove pos marker
jump NO_DUPS
REMOVED:
cursordown jtrue LOOP * MOVE down to next line and loop
EOF:
begfile
togglewordwrap toggleinsert
*
* 103 bytes Wed 05-27-1992 11:35:35 (TH @9, to ALL #3/854-56)
* 104 bytes Wed 05-27-1992 19:15:53 (TH @9, added endline *|)
*
* ----------------------------------------------------------------------
* @(0) Mark Non-Duplicated Lines In File
* ----------------------------------------------------------------------
* This macro marks all non-empty non-duplicated lines in the current
* file with '*▐' in column 77 and leaves lines that are duplicated in
* the file unmarked. Since the macro sets Insert Off, any
* non-duplicated lines having text in columns 77 and 78 will be
* overwritten. This macro operates in reverse of @9.
@0 macrobegin
begfile
setscreenoff setsoundoff
setwordwrapmode togglewordwrap
setinsmode toggleinsert
* ---------------------- Bypass All Empty Lines ----------------------*
LOOP:
endline begline jtrue NOT_EMPTY
cursordown jtrue LOOP
NOT_EMPTY:
* ------- Copy Line To Scrap For Find, Insert Position Marker -------*
unmarkblock makectrofscreen
begline markcolumn endline
cursorleft markcolumn
copy
cursorright #173 * Insert starting pos marker
* --------------------- Find/Mark Duplicate Lines --------------------*
FIND_DUPS:
find paste return * Search for first duplicate line
delline return
jfalse NO_DUPS * Del markers when no more dup's
isbegline jfalse FIND_DUPS * Bypass multiple dup's in same line
pasteover gotoblockend * Paste block over line with 'find'
endline jtrue FIND_DUPS * Test true if no dup line
gotocolumn "77" return "*▐"
HAS_DUPS:
endline * Pos for find *|
find #173 return "b" return
jfalse MARK_DUP
delch * Remove pos marker
jump HAS_DUPS
MARK_DUP:
gotocolumn return "*▐"
jump REMOVED * MOVE down to next line and loop
NO_DUPS:
endline * Pos for find *|
find #173 return "b" return
jfalse REMOVED
delch * Remove pos marker
jump NO_DUPS
REMOVED:
cursordown jtrue LOOP * MOVE down to next line and loop
EOF:
begfile
* -------- Mark Non-Duplicated Lines, Unmark Duplicated Lines --------*
UNDO:
markline
gotocolumn "77" return
cursorleft * Pos for find
find "*▐" return
"L" return jfalse MARK
delch delch * Delete all markers
jump DOWN
MARK:
cursorright "*▐" * Line is non-dup, mark it
DOWN:
cursordown jtrue UNDO * Loop 'til done
begfile unmarkblock
togglewordwrap toggleinsert
*
* 139 bytes Wed 05-27-1992 12:01:15 (TH @0, to ALL #3/854-56)
* 141 bytes Wed 05-27-1992 19:15:57 (TH @0, added endline *|)
*
* ----------------------------------------------------------------------
* @(f8) Retain Only All Non-Duplicated Lines In File
* And Remove All Duplicated Lines,
* Duplicated Lines Retained In Nul File,
* Uses Scratch Buffers Instead of UnKill In @8
* ----------------------------------------------------------------------
* This macro retains only all non-empty non-duplicated lines in the
* current file and removes all lines that are duplicated. Duplicated
* lines are retained in the Nul file for viewing. The scratch buffer
* technique is used here, so the number of duplicated lines that can be
* removed and viewed in the Nul file is practically unlimited.
@f8 macrobegin
begfile
setscreenoff setsoundoff
setwordwrapmode togglewordwrap
* ------------ Clear Scrap Buffer, Insure Nul Not In Ring ------------*
endline markcolumn * Make empty block
storescrbuff "d" return * Clear buff "d"
editfile "nul" return quit * Quit Nul for ring pos
gotoblockbeg * Return to file to remove dup's
* ---------------------- Bypass All Empty Lines ----------------------*
LOOP:
endline begline jtrue NOT_EMPTY
cursordown jtrue LOOP
NOT_EMPTY:
* ------- Copy Line To Scrap For Find, Insert Position Marker -------*
unmarkblock makectrofscreen
begline markcolumn endline
cursorleft markcolumn
copy
cursorright #173
* -------------------- Find/Delete Duplicate Lines -------------------*
FIND_DUPS:
find paste return * Search for first duplicate line
delline return
jfalse NO_DUPS * Del markers when no more dup's
isbegline jfalse FIND_DUPS * Bypass multiple dup's in same line
pasteover gotoblockend * Paste block over line with 'find'
endline jtrue FIND_DUPS * Test true if no dup line
appendscrbuff "d" return * If false, it's a dup, append to buff
delline * If false, it's a dup, delete it
cursorup * In case next line is dup also
HAS_DUPS:
endline * Pos for find
find #173 return "b" return
jfalse REMOVE_DUP
delch * Remove pos marker
jump HAS_DUPS
REMOVE_DUP:
markline *|
appendscrbuff "d" return *|
delline * Line had at least 1 dup, delete it
cursorup * In case next line is dup also
jump LOOP
NO_DUPS:
endline * Pos for find
find #173 return "b" return
jfalse REMOVED
delch * Remove pos marker
jump NO_DUPS
REMOVED:
cursordown jtrue LOOP * MOVE down to next line and loop
begfile
* ------------- Get Duplicated Lines To Nul For Viewing -------------*
editfile return
getscrbuff "d" return
* ----------------- Remove All Empty Lines From Nul -----------------*
CHK4BLANK: endline begline Jtrue NOTBLANK delline Jtrue CHK4BLANK
NOTBLANK: endpara cursordown Jtrue CHK4BLANK
* ------------- Position To View Duplicate Lines Deleted -------------*
makebotofscreen cursorup
setscreenon
pause * Press <enter> to return to file w/o dup's
prevfile unmarkblock
togglewordwrap
*
* 162 bytes Wed 05-27-1992 19:15:48 (TH @8, uses UnKill)
* 136 bytes Wed 05-27-1992 19:40:42 (TH @f8, uses scratch buffers)
* 141 bytes Wed 05-27-1992 20:24:18 (TH @f8, added appendscrbuff *|)